Fractionally Predictive Spiking Neurons

Recent experimental work has suggested that the neural firing rate can be interpreted as a fractional derivative, at least when signal variation induces neural adaptation. Here, we show that the actual neural spike-train itself can be considered as the fractional derivative, provided that the neural signal is approximated by a sum of power-law kernels. A simple standard thresholding spiking neuron suffices to carry out such an approximation, given a suitable refractory response. Empirically, we find that the online approximation of signals with a sum of power-law kernels is beneficial for encoding signals with slowly varying components, like long-memory self-similar signals. For such signals, the online power-law kernel approximation typically required less than half the number of spikes for similar SNR as compared to sums of similar but exponentially decaying kernels. As power-law kernels can be accurately approximated using sums or cascades of weighted exponentials, we demonstrate that the corresponding decoding of spike-trains by a receiving neuron allows for natural and transparent temporal signal filtering by tuning the weights of the decoding kernel.Comment: 13 pages, 5 figures, in Advances in Neural Information Processing 201

A learning rule that explains how rewards teach attention

Many theories propose that top-down attentional signals control processing in sensory cortices by modulating neural activity. But who controls the controller? Here we investigate how a biologically plausible neural reinforcement learning scheme can create higher order representations and top-down attentional signals. The learning scheme trains neural networks using two factors that gate Hebbian plasticity: (1) an attentional feedback signal from the response-selection stage to earlier processing levels; and (2) a globally available neuromodulator that encodes the reward prediction error. We demonstrate how the neural network learns to direct attention to one of two coloured stimuli that are arranged in a rank-order. Like monkeys trained on this task, the network develops units that are tuned to the rank-order of the colours and it generalizes this newly learned rule to previously unseen colour combinations. These results provide new insight into how individuals can learn to control attention as a function of reward contingenc

Tuning in the association layer in the probabilistic classification task.

<p><b><i>A</i></b>, Trials were subdivided in quintiles based on the log-likelihood ratio of the evidence favoring one target. Average activations of the four memory units of a trained model network (top; 100,000 trials) and LIP neurons (bottom, from [<a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1004060#pcbi.1004060.ref005" target="_blank">5</a>]) depend on the log-likelihood ratio. <b><i>B</i></b>, Left, Average synaptic weights between input units representing symbols and an example memory unit are strongly correlated (<i>ρ</i>≈1, <i>p</i><10^-6) with true symbol weights. Right, Subjective weights assigned by a monkey as estimated from the performance data (from [<a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1004060#pcbi.1004060.ref005" target="_blank">5</a>]). <b><i>C</i></b>, Histogram of Spearman correlations between average synaptic weights for symbols and true symbol weights for 396 memory units (AuGMEnT trained 99 of 100 simulated networks to criterion). Note that there are also units with zero correlation that do not contribute to the mapping of the symbols onto <i>Q</i>-values. These units were accompanied by other association units with stronger correlations.</p

Model Architecture.

<p><b><i>A</i></b>, The model consists of a sensory input layer with units that code the input (instantaneous units) and transient units that only respond when a stimulus appears (on-units) or if it disappears (off-units). The association layer contains regular units (circles) with activities that depend on instantaneous input units, and integrating memory units (diamonds) that receive input from transient sensory units. The connections from the input layer to the memory cells maintain a synaptic trace (sTrace; blue circle) if the synapse was active. Units in the third layer code the value of actions (Q-values). After computing feed-forward activations, a Winner-Take-All competition determines the winning action (see middle panel). Action selection causes a feedback signal to earlier levels (through feedback connections , see middle panel) that lays down synaptic tags (orange pentagons) at synapses that are responsible for the selected action. If the predicted <i>Q</i>-value of the next action <i>S′</i> (<i>Q_S′</i>) plus the obtained reward <i>r</i>(<i>t</i>) is higher than <i>Q_S</i>, a globally released neuromodulator <i>δ</i> (see <a href="http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1004060#pcbi.1004060.e038" target="_blank">eq. (17)</a>) interacts with the tagged synapses to increase the strength of tagged synapses (green connections). If the predicted value is lower than expected, the strength of tagged synapses is decreased. <b><i>B</i></b>, Schematic illustration of the tagging process for regular units. FF is a feed-forward connection and FB is a feedback connection. The combination of feed-forward and feedback activation gives rise to a synaptic tag in step ii. Tags interact with the globally released neuromodulator <i>δ</i> to change the synaptic strength (step iv,v). <b><i>C</i></b>, Tagging process for memory units. Any presynaptic feed-forward activation gives rise to a synaptic trace (step ii; sTrace—purple circle). A feedback signal from the <i>Q</i>-value unit selected for action creates synaptic tags on synapses that carry a synaptic trace (step iv). The neuromodulator can interact with the tags to modify synaptic strength (v,vi).</p

Varying the size of the association layer.

<p><b><i>A</i></b>, Scaling with unchanged learning parameters <i>β</i> and <i>λ</i>. Left, convergence rate (proportion of 100 networks that learned the saccade/antisaccade task). Error bars denote 95% confidence intervals. Right, median convergence speed (number of trials to criterion). <b><i>B</i></b>, Left, convergence rates with adjusted learning parameters. Bar shading indicates parameter setting (see legend in right panel). Right, median convergence speed with optimized parameters.</p

Model parameters.

<p>Model parameters.</p